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The present invention relates to apparatus and methods for re-embedding 
information in multimedia signals, such as audio, video or data signals. 

5 Watermarking of multimedia signals is a technique for the transmission of 

additional data along with the multimedia signal. For instance, watermarking techniques can 
be used to embed copyright and copy control information into audio signals. 

The main requirement of a watermarking scheme is that it is not observable 
(i.e. in the case of an audio signal, it is inaudible) whilst being robust to attacks to remove the 

10 watermark from the signal (e.g. removing the watermark will damage the signal). It will be 
appreciated that the robustness of a watermark will normally be a trade off against the quality 
of the signal in which the watermark is embedded. For instance, if a watermark is strongly 
embedded into an audio signal (and is thus difficult to remove) then it is likely that the 
quality of the audio signal will be reduced, if one tries to remove it without the knowledge of 

15 the underlying technique and the secret key. 

The altering of a watermark by increasing the amount of embedded 
information in a watermark signal is known. In such instances, an extra watermark sequence 
is added to an existing watermarked signal. This is, for instance, implied in the 4C 12 bit 
watermark specification. A copy of this specification can be found at 

20 http://www.4centity.com/data/tech/4cspec.pdf. 

It will be appreciated that when the watermark information needs to be 
changed repeatedly, such an approach not only degrades the quality of the original 
information signal (as additional watermark signals are added to change the payload) but also 
collisions between the individual embedded watermarks significantly degrade the watermark 

25 robustness. For example, certain applications of copyright require that the copyright 
information embedded in a signal is changed repeatedly in order to assert proper copy 
control. 
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It is an object of the present invention to provide a watermarking scheme that 
substantially addresses at least one of the problems of the prior art, whether referred to herein 
or otherwise. 



5 

In a first aspect, the present invention provides a method of processing a 
multimedia signal comprising a watermark signal, the method comprising the steps of: 
removing at least a portion of an original watermark signal; and adding a new watermark 
signal to the multimedia signal so as to form a new watermarked multimedia signal. 
1 0 Preferably, said original watermark signal is removed by applying a negative 

version of said original watermark signal to the multimedia signal. 

Preferably, the method further comprises the step of determining the value of 
at least one of the parameters used to embed the original watermark in the multimedia signal. 
Preferably, said parameter is utilized to remove at least a portion of said 
1 5 original watermark signal. 

Preferably, said new signal is embedded in the multimedia signal using said 
embedding parameters having said determined value. 

Suitably, said new signal is embedded in the multimedia signal using said 
embedding parameters having values other than the said determined values. 
20 Preferably, said parameter comprises at least one of: embedding strength, 

synchronization information, time offset, time-scaling, an amount of a circular shift of a 
sequence, and a watermark symbol period. 

Suitably, all of said original watermark is removed, and the new signal 
comprises a new watermark signal. 
25 Preferably, said watermark comprises at least two sequences of values, at least 

one sequence of values being removed as said portion of the watermark signal, so as to leave 
at least one remaining sequence of values from the original watermark signal; and wherein 
said new signal comprises at least one further sequence of values which together with said 
remaining sequence forms a new watermark signal. 
30 Preferably, all of said sequences of values are formed from a single sequence 

of values which has been circularly shifted by different amounts. 

Preferably, said removed portion of the watermark signal was embedded with 
a predetermined strength into said multimedia signal, the method comprising the step of 
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embedding the new signal into the multimedia signal with preferably the same predetermined 
strength. 

Preferably, the embedding strength is such that the degradation in the quality 
of the new watermarked multimedia signal is perceptible but not annoying. 
5 Preferably, at least one of the original watermark signal and the new 

watermark signal comprises a smoothly varying signal formed by applying a window shaping 
function to a sequence of values, the integral over the window shaping function being zero. 

Preferably, the window shaping function has a bi-phase behavior. 

Preferably, the bi-phase window comprises at least two Harming windows of 
10 opposite polarities. 

In another aspect, the present invention provides a computer program arranged 
to perform any of the methods described above. 

In a further aspect, the present invention provides a record carrier comprising a 
computer program described above. 
15 In another aspect, the present invention provides a method of making available 

for downloading a computer program as described above. 

In a further aspect, the present invention provides an apparatus for processing 
a multimedia signal comprising a watermark signal, the apparatus comprising: a deletion unit 
arrange to remove at least a portion of the watermark signal; and an embedder arranged to 
20 add a new signal to the multimedia signal so as to form a new watermark signal. 

Preferably, the apparatus further comprises a detector arranged to detect at 
least one value of a parameter of said watermark signal. 

Preferably, the apparatus comprises a receiver of a multimedia signal 
comprising an apparatus. 

25 



For a better understanding of the invention, and to show how embodiments of 
the same may be carried into effect, reference will now be made, by way of example, to the 
accompanying diagrammatic drawings in which: 
30 Figure 1 is a diagram illustrating a generalized watermark re-embedding 

apparatus according to a first embodiment of the present invention; 

Figure 2 shows a schematic diagram of one type of watermark embedder; 

Figure 3 is a schematic diagram showing the details of the payload adding and 
conditioning circuit (unit 630a of figure 2) used for embedding a primary watermark payload; 
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Figure 4 is a schematic diagram illustrating a watermark detector that can be 
used in one preferred embodiment of the invention; 

Figure 5 is a schematic diagram showing the details of watermark generator 
unit 650 according to a preferred embodiment; 
5 Figure 6 is a diagram illustrating a watermark embedder apparatus according 

to a preferred embodiment; 

Figure 7 shows a signal portion extraction filter 

Figure 8 shows the payload adding/changing and watermark conditioning 
stage used for re-embedding a watermark; 
1 0 Figure 9 is a diagram illustrating the details of the watermark conditioning 

apparatus H c of figure 8, including charts of the associated signals at each stage; 

Figure 10 shows a typical correlation function corresponding to the original 
payload of a watermark signal received by the apparatus of figure 1 and figure 4; 

Figure 1 1 shows a correlation function after removal of one of the two 
1 5 sequences forming the original watermark; and 

Figure 12 shows a correlation function of a new watermark having a new 
payload, formed by adding a second watermark sequence to the signal having the correlation 
function shown in figure 10. 

20 

Figure 1 illustrates a re-embedding apparatus 600 according to a first 
embodiment of the present invention. The apparatus includes an input 602 arranged to 
receive a watermarked information signal v W Two copies of the input signal y ' 0 u are 
formed, with one copy going to a delay unit 614, and the other being input to a detector 640. 

25 The detector 640 is arranged to detect an estimate w '„id (output 612) of the 

watermark w oH that is embedded within the received signal y ' 0 w> and to estimate the control 
parameters (output 609) needed for changing the watermark payload. 

Information relating to the detected watermark w ' 0 id is then passed to the 
watermark generator 650 and the watermark embedder 620. The information passed to the 

30 watermark generator can be a complete copy of the extracted watermark w ' 0 id, or 

alternatively sufficient information (such as, for watermarks comprising two circularly 
shifted sequences of a single series of values, the circular shift d old ) to allow the watermark 
generator to generate a copy w old ofw' 0 , d . The output 608 of the watermark generator, also 
denoted by w oW , is preferably an error corrected version of the extracted watermark w ' M 
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The watermark generator 650 additionally generates a new watermark signal 
Wnew (output 618), for adding to the original information signal y' 0 id- 

The delay unit 614 acts to delay the input signal y ' 0 id whilst the intervening 
operations are carried out by the detector 640, and the watermark generator 650. The delayed 
5 signal y' 0 id is then passed (output 607) to the embedder 620. 

In the watermark embedding unit 620, the old watermark signal w oU is 
removed from the signal y ' 0 id, and the new watermark w new added to y ' oid so as to form an 
information signal y new containing the new watermark w new . 

The above embodiment represents a particular generalized implementation of 
10 the present invention. Below is described a particular watermark re-embedding scheme in 
accordance with a further embodiment of the present invention for use in conjunction with a 
particular watermarking scheme. 

Such a watermark can for example be embedded using the apparatus shown in 
figures 2 and 3. 

1 5 Figure 2 shows an embedding apparatus 1 00 arranged to receive a host 

multimedia signal x at input 1 20, and output a watermarked version of the received signal 
(yoid) at output 132. 

The apparatus 100 receives two sequences of values iw 0 u[k] and Wre/fkJ) at 
input 1 10 and 1 12. The combination of these two sequences of values is used to produce the 

20 watermark signal w 0 . Each of the two sequences of values is a circularly shifted version of an 
original sequence of values. W 0 \d is the original sequence w s circularly shifted by d 0 u, and w re f 
the original sequence w s circularly shifted by d re f i.e. w a i d is hence a circularly shifted version 
of w r ef. The original sequence of values can, for instance, be generated using a Random 
Number Generator (RNG) using a predetermined seed value S. 

25 As shown in Figure 3, the payload adding and conditioning circuit 630a 

receives w 0 id[k] and w re /fkj at respective inputs 1 10 and 1 12. Each sequence is multiplied by 
a respective sign bit r 0 u and r re /(received at inputs 603 and 604) using multipliers 631 (where 
r 0 id and /v^ are respectively +1 or -1, and remain constant for any given payload). The 
resulting signals are then passed through signal conditioning units 632, which act to convolve 

30 each of the values within each sequence with a window shaping function of the period T s . 
This produces smoothly varying output sequences w 0 u[nj and w re /{nj. Operation of a signal 
conditioning unit 632 is described later in more detail with reference to Figure 9. 
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Signal w re/ fnj is then delayed by delay unit 634 by a predetermined delay T r (where T r is less 
than T s , and preferably, T r =TJ4). Adder 635 then adds w 0 u[n] and the delayed version of 
WreffnJ, the result being the watermark signal w 0 [n] which is output from the circuit 630 at 
output 623. 

Referring back to Figure 2, the received host signal x is split into two copies, a 
first copy going to adder 130, and a second copy going to multiplication unit 124. The copy 
sent to multiplication unit 124 may be filtered using a band pass filter (not shown) so as to 
form the signal x b at input 122 to the multiplier 124. 

W 0 [n] is multiplied with the possibly filtered version XbfnJ of the host signal 
xfnj, scaled by gain factor a and added back to the host signal xfn] to generate the 
watermark signal y 0 u[n] given by 

^[»] = 4»] + <a»,[BKW. (1) 
The gain factor a is set by the variable gain device 128, and in the example shown is set in 
dependence upon the signal from the signal analyzer 126 which samples the host signal x 
using a psychoacoustic model. Such a model is, for instance, described in the paper by 
E.Zwicker, "Audio Engineering and Psychoacoustics: Matching signals to the final receiver, 
the Human Auditory System", Journal of the Audio Engineering Society, Vol. 39, pp. 
Vol.1 15-126, March 1991. The gain factor a is normally chosen so as to minimize the impact 
of the watermark signal on the host signal quality. 

Such a watermarking scheme is characterized in that, during detection the 
watermarked signal y 0 u generates two correlation peaks that are separated by pL oU (see figure 
10), where the value pL oU is at least part of the watermark payload, and may be defined as 

pL oU ^-d oU \mo^ L y(^ (2) 

where c? re /and d 0 id <= [0, 2^-1] are the relative positions of the correlation peaks as seen in 
figure 10, and Lw is the length (number of symbols or values) of each of the two watermark 
sequences. 

In addition to pL 0 id, extra information is also encoded by changing the relative 
signs of the embedded watermarks. In the detector, this is seen as a relative sign r sign between 
the correlation peaks. It will be seen that r sign can take four possible values, and may be 
defined as: 
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^ J-Hl+f+' .tort. (3) 

where /wand /u ref are the signs of the correlation peaks, and correspond to the sign bits r 0 u 
and r re /in figure 3, respectively. The overall watermark payload pL w is then given as a 
combination of r sign and pL 0 u: 

pL w ={r^,pL old ). (4) 



5 The maximum information (I max ), in number of bits, that can be carried by a watermark 
sequence of length i w is thus given by: 



log 2 4- \W 2 bits (5) 



Below is described a watermark re-embedding apparatus suitable for use in 
conjunction with the watermark embedder shown in figure 2 and the related watermarking 

10 scheme. In this particular embodiment, the embedding apparatus is arranged to only partially 
remove the watermark, and replaces it with new information so as to change the watermark 
payload. Figure 1 illustrates the different functional blocks of the re-embedder. 

Figure 4 shows a block diagram of a watermark detector 640 used to extract 
the watermark embedding parameters that are needed for the re-embedding process. The 

1 5 detector consists of four major stages: (a) the watermark symbol extraction stage (200), (b) 
the buffering and interpolation stage (300), (c) the correlation and decision stage (400) and 
(d) the control signal generating stage (500). 

In the symbol extraction stage (200), the received watermarked signal y' 0 u[n] 
is processed to generate multiple (N b ) estimates of the watermarked sequence, which are 

20 multiplexed into the signal w e fmj. These estimates of the watermark sequence are required to 
resolve (compensate for) any time offset that may exist between the embedder and the 
detector, so that the watermark detector can synchronize to the watermark sequence inserted 
in the host signal. 

In the buffering and interpolation stage (300), these estimates are de- 

25 multiplexed into Nb separate buffers. An interpolation is subsequently applied to each buffer 
to resolve (compensate for) possible timescale modifications that may have occurred. For 
instance, a drift in sampling (clock) frequency may result in a stretch or shrink in the time 
domain signal (i.e. the watermark may have been stretched or shrunk). 
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In the correlation and decision stage (400), the content of each buffer is 
correlated with the reference watermark and the maximum correlation peaks are compared 
against a threshold to determine the likelihood of whether the watermark is indeed embedded 
within the received signal y ' 0 id[nJ. 

In the control signal generating unit (500), the detection truth value, the 
corresponding watermark sequence, its buffer index, and the value of the new payload pL^ 
(input 616) are combined to generate parameters that are needed to changing the watermark 
payload. The outputs (609, 612) of the control signal generator are passed on to the 
watermark generator 650 and the watermark embedding unit 620. 

Details of the watermark generating unit 650 are shown in figure 5. In this 
unit, Wnew and w 0 w are generated using parameter information obtained from the detector 640 
(inputs 609 and 612). 

In one preferred embodiment, the watermark sequences w 0 i d and w new can be 
generated as follows. Firstly a finite length, preferably zero mean and uniformly distributed 
random sequence w s is generated using a random number generator 651 with an initial seed S. 
It will be appreciated that this initial seed S is preferably the same as that used in generating 
Woia during the first embedding. This results in the sequence of length L w 
w s [k] e [-1,1], for k=0,l,2, .... L w -1 (6) 

Then the sequence w s is circularly shifted by the amounts d 0 u and d new using 
the circularly shifting units 653a and 653b to obtain the random sequences w M and w new , 
respectively. It will be appreciated that these two sequences (w oW and w new ) are effectively a 
first sequence and a second sequence, with the second sequence being circularly shifted with 
respect to the first. These two sequences are then passed on to the watermark embedding unit 
620 (outputs 608, 618). 

Details of a watermark embedding unit 620 as part of the watermark re- 
embedding apparatus 600, and suitable for use with the considered particular watermarking 
scheme, is shown in figure 6. 

The host signal y ' M (containing at least an initial watermark sequence w 0 id) is 
provided at input 607 of the apparatus. The host signal y ' M is passed in the direction of 
output 610 via the delay unit 629 and the adder 626. However, a replica y b of the host signal 
y' M (input 628) is split off in the direction of the multiplier 624, for carrying the new 
watermark information. 

The multiplier 624 is utilized to calculate the product of the watermark 
altering signal w di fr and the replica signal^. The watermark altering signal w d ifr is obtained 
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from the payload changing and watermark conditioning apparatus 630b, and derived from the 
watermark random sequences w„id and w new (inputs 608 and 618, respectively), which are 
input to the payload changing and watermark conditioning apparatus. 

The resulting product, Wdiflyb is then passed via a gain controller 625 to the 
5 adder 626. The gain factor a applied by the controller 625 controls the trade off between the 
audibility and the robustness of the watermark. It may be a constant, or variable in at least 
one of time, frequency and space. The apparatus in figure 6 shows that, when a is variable, it 
can be automatically adapted via a control signal 609 obtained from the watermark detector 
unit 640. 

10 In another preferred embodiment, the gain factor a can be independently 

controlled via a signal analyzer based upon the properties of the host signal y„id- In the latter 
case, the gain a is automatically adapted, preferably so as to minimize the impact on the 
signal quality, according to a properly chosen perceptibility cost-function, such as a 
psychoacoustic model of the human auditory system (HAS). The same model may be used 

15 as that utilized to control the adaptive gain factor used to control the embedding strength of 
the original watermark signal (see Figure 2 and associated text). 

In Figure 6, the resulting watermark audio signal y new is then obtained at the 
output 610 of the embedding apparatus 620 by adding an appropriately scaled version of the 
product of Wdiff and y b to the host signal: 

jwM = you ["3 + ™>diffWybl"\ * *M+ ™>c Wn, M - M 
20 where w c is derived from w re /-and w new in the same way as w 0 was derived from w re /and w 0 u 
in figure 3. 

Preferably, the parameters of the watermark Wdigfn] are chosen such that when 
multiplied with it predominantly modifies the short time envelope of^. 

Figure 7 shows one preferred embodiment in which the input 628 to the 
25 multiplier 624 in figure 6 is obtained by filtering the host signal y ' oW using a filter H in the 
filtering unit 615. Preferably, the filter H is a linear phase band-pass filter characterized by its 
lower cut off frequency/i and upper cut off frequency fy- Preferably, the filter has the same 
properties as the filter utilized to extract x b from x in the embedder 100. 

In figure 8, the details of the payload changing and watermark-conditioning 
30 unit 630b are shown. In this particular unit, the watermark signals w 0 w and w new are combined 
to generate the multi-bit watermark altering signal w di ff. The watermark altering signal w diff , 



WO 03/102947 PCT7IB03/02319 

10 

when combined with/ oW , generates a watermarked signal with a payload corresponding to 

Wnew- 

The sequences w oW and w new are first multiplied with a respective sign bit r 0 u 
and r new in the multiplying units 654a and 654b. The respective values of r„i d and r new are 
5 derived from the detection unit 640 and are passed on via the control input 609. The values of 
r 0 i d and r new remain constant (typically at either +1 or-1), and only change when the payload 
of the watermark is changed. 

The difference w d jff[k] between the signed sequences w 0 u and w new is 
calculated using adder 635 to add the negative of the old watermark sequence w oW to the 
1 0 positive of the new sequence w new . The result w^is then passed through the conditioning 
stage to generate the slowly varying multi-bit watermark Wdirfn]. 

Figure 9 shows details of the watermark conditioning apparatus 632 used in 
the payload adding/changing and watermark conditioning apparatus 630. In the case of re- 
embedding, the watermark random sequence w diff = w new -w 0 u is input to the conditioning 
15 apparatus 632. 

In the conditioning circuit, the watermark signal sequence w di gk] is first 
applied to the input of an up-sampler 180. Chart 181 illustrates one of the possible sequences 
wnffZS a sequence of values of random numbers between +1 and -1, with the sequence being 
of length L w . The up-sampler adds (7>i) zeros between each sample so as to raise the 

20 sampling frequency by the factor T s . T s is referred to as the watermark symbol period and 
represents the span of the watermark symbol in the audio signal. In the case of the received 
signal y ' 0 u having undergone a time scaling compared with the transmitted signal y 0 id, T s is 
replaced with appropriately scaled sampling factor T new that takes into account the scaling 
effect. Chart 183 shows the results of the signal illustrated in chart 181 once it has passed 

25 through the up-sampler 1 80. 

A window shaping function sfnj, such as a bi-phase window, is then 
convolved with the up-sampled signal w i[n] so as to convert it into a slowly varying narrow- 
band signal Wdifffn], whose behavior for the w d iff[k] sequence of chart 181 is as shown in chart 
185. 

30 Chart 1 84 shows a typical bi-phase window shaping function. The window 

shaping function has support in the interval 0 to T s only. The window function is applied to 
the watermark sequence in order to produce a smoothly varying signal, so as to minimize the 
decrease in the quality of the host signal. 
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Below is described in more detail the operation of the detection apparatus 
(200, 300, 400, 500) shown in figure 4. 

In the watermark symbol extraction stage 200 shown in figure 4, the incoming 



watermark signal y ' oU fnJ is input to the signal conditioning Rlter H b (2 10). This filter 210 is 
5 typically a band pass filter and has the same behavior as the corresponding filter H (615) 
shown in figure 7. The output of the filter H b is y ' b fnJ, and assuming linearity within the 
transmission channel, it follows from equation (1) 



Note that when no filter is used in the embedder (i.e., when H=l) then H b in the detector can 
also be omitted, or it can still be included to improve the detection performance. If H b is 
1 0 omitted, then y ' b in equation (8) is replaced with y 'old- The rest of the processing is the same. 



embedder and the detector (i.e. no offset and no change in timescale), and that the audio 
signal is divided into frames of length T s , and that y' b>m [n] is the n-th sample of the m-th 
frame of the filtered signal y' b [n]. It should be noted that if there is not perfect synchronism 
15 between the embedder and the detector, then any deviation can be compensated for within the 
buffering and interpolation stage 300 utilizing techniques known to the skilled person e.g. 
iteratively searching through all possible scale and offset modifications until a best match is 
achieved. 

The energy EfmJ corresponding to they' b , m [n] frame is: 



20 where S[n] is the same window shaping function used in the watermark conditioning circuit 
of figure 9. A person skilled in the art will appreciate that equation 9 represents a matched 
filter receiver, and is the optimum receiver when the symbol period is perfectly synchronized. 
Not withstanding this fact, from now on, we set S[n]=l in order to simplify subsequent 
explanations. 

25 Combining this with equation 8, it follows that: 



y' b [n]*(\ + aw c [n))x b [n] 



For simplification, it is assumed that there is perfect synchronism between the 




(9) 




Id 



T,-l 



(10) 



where w e [m] is the m-th extracted watermark symbol, and contains N b time-multiplexed 
estimates of the embedded watermark sequences. Solving for w e [m] in equation 10 and 
ignoring higher order terms of a, gives the following approximation: 
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(11) 



In the watennark extraction stage 200 shown in figure 4, the output y ' b fnJ of 
the filter H b is provided as an input to a frame divider 220, which divides the audio signal 
into frames of length T s i.e. into y W"7> with the energy calculating unit 230 then being used 
to calculate the energy corresponding to each of the framed signals as per equation (9). The 
output of this energy calculation unit 230 is then provided as an input to the whitening stage 
H w (240) which performs the function shown in equation 1 1 so as to provide an output w e [mj. 

It will be realized that the denominator of equation 1 1 contains a term that 
requires knowledge of the host (original) signal x. As the signal x is not available to the 
detector, it means that in order to calculate w e [mj then the denominator of equation 1 1 must 
be estimated. 

Below is described how such an estimation can be achieved for a bi-phase 
window shaping function, but it will be appreciated that the teaching could be extended to 
other window shaping functions. 

It will be seen by examination of the bi-phase window function shown in 
figure 9 (chart 184), that when the envelope of an audio frame is modulated with such a 
window function, the first and the second halves of the frame are scaled in opposite 
directions. In the detector, this property is utilized to estimate the envelope energy of the host 
signal y'c d . 

Consequently, within the detector, the audio frame is first sub-divided into two 
halves. The energy functions corresponding to the first and second half frames are hence 
given by 

T,n-i i , 



E 2 [m]= t\y\»[nf ( 13 > 

n=T,/2 

respectively. As the envelope of the original audio is modulated in opposite directions within 
the two sub-frames, the original audio envelope can be approximated as the mean of EjfmJ 
and Ei[m]. 
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Further, the instantaneous modulation value can be taken as the difference 
between these two functions. Thus, for the bi-phase window function, the watermark w e [m] 
can be approximated by: 



This output w e fmj is then passed to the buffering and interpolation stage 300, where the 
5 signal is de-multiplexed by a de-multiplexer 310, buffered in buffers 320 of length U so as to 
resolve any lack of synchronism between the embedder and the detector, and interpolated 
within the interpolation unit 330 so as to compensate for any time scale modification between 
the embedder and the detector. Such compensation can utilize known techniques, and hence 
is not described in any more detail within this specification. 

1 0 During detection, in order to maximize the accuracy of the watermark 

detection, the watermark detection process is typically carried out over a length of received 
signal y ' 0 u[n] that is 3 to 4 times that of the watermark sequence length. Thus each 
watermark symbol to be detected can be constructed by taking the averages of several 
symbols. This averaging process is referred to as smoothing, and the number of times the 

1 5 averaging is done is referred to as the smoothing factor s f . Thus, the detection window length 
L D is the length of the audio segment (in number of samples) over which a watermark 
detection truth-value is reported. Consequently, L D =sjL w T s , where T s is the symbol period and 
L w the number of symbols within the watermark sequence. Typically, the length (Lt) of each 
buffer 320 within the buffering and interpolation stage is Lb~s/L w . 

20 As shown in figure 4, outputs (w D] , w D 2, - w DNb ) from the buffering stage are 

passed to the interpolation stage and, after interpolation, the outputs fwn, ~wn, — of this 
stage, which correspond to the different estimates of the correctly re-scaled signal, are passed 
to the correlation and decision stage. If it is believed that no time scaling compensation is 
required, the values (w D i, wd2, - *fz>W can be passed directly to the correlation and decision 

25 stage 400 i.e. the interpolation stage 330 can be omitted from the apparatus. 



corresponding to each estimate is then applied to the maximum detection unit 420 which 
determines which two estimates provided the best fits for the circularly shifted versions w 0 u 
30 and vtv^of the reference watermark. The correlation values (the peak amplitudes and 




(14) 



The correlator 410 calculates the correlation of each estimate W[j,j=], ...,Nb 
with respect to the reference watermark sequence w s [kj. Each respective correlation output 
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positions) for these estimate sequences are passed to the threshold detector and payload 
extractor unit 430. 

In another output of the correlation stage 410, the watermark sequences and 
the buffer indices corresponding to the two best fits for the circularly shifted versions w 0 u 
5 and w re /of the reference watermark are passed on to the control signal generating unit 500. 

If the interpolation stage is omitted, alternatively the correlator 410 calculates 
the correlation of each estimate w D j,j=l, ...,N b with the reference watermark sequence w s fk] 
and the results are passed on for subsequent processing to the units 420 and 430 as outlined 
in the above paragraph. 
1 0 The threshold detector and payload extractor unit 430 may be utilized to 

extract the payload (e.g. information content) from the detected watermark signal. Once the 
unit has estimated the two correlation peaks cLi and chz that exceed the detection threshold, 
the distance pL between the peaks (as defined by equation (2)) is measured. Next, the signs 
Hi and fj.2 of the correlation peaks are determined, and hence recalculated from equation 
1 5 (3). The overall watermark payload may then be calculated using equation (4). 

For instance, it can be seen in Figure 10 that pL old is the relative distance 
between the two peaks. Both peaks are positive i.e. p t - +1, and & - +1. From equation (3), 
r S jgn = 3. Consequently, the payload pL w = <3,pL 0 id>. 

The reference watermark sequence w s used within the detector corresponds to 
20 (a possibly circularly shifted version of) the original watermark sequence applied to the host 
signal. For instance, if the watermark signal was calculated using a random number generator 
with seed S within the embedder, then equally the detector can calculate the same random 
number sequence using the same random number generation algorithm and the same initial 
seed so as to determine the watermark signal. Alternatively, the watermark signal originally 
25 applied in the embedder and utilized by the detector as a reference could simply be any 
predetermined sequence. 

Figure 10 shows a typical shape of a correlation function as output from the 
correlator 410. The horizontal scale shows the correlation delay (in terms of the sequence 
bins). The vertical scale on the left-hand side (referred to as the confidence level cL) 
30 represents the value of the correlation peak normalized with respect to the standard deviation 
of the (typically normally distributed) correlation function. 

As can be seen, the typical correlation is relatively flat with respect to cL, and 
centered about cL = 0. However, the function contains two peaks, which are separated by 
pLoid (see equation 2) and extend upwards to cL values that are above the detection threshold 
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when a watermark is present. When the correlation peaks are negative, the above statement 
applies to their absolute values. 

A horizontal line represents the detection threshold. The detection threshold 
value controls the false alarm rate. 
5 Two kinds of false alarms exist: the false positive rate, defined as the 

probability of detecting a watermark in non watermarked items, and the false negative rate, 
which is defined as the probability of not detecting a watermark in watermarked items. 
Generally, the requirement of the false positive alarm is more stringent than that of the false 
negative. 

1 o After each detection interval, the detector determines whether the original 

watermark is present or whether it is not present, and on this basis output a "yes" or a "no" 
decision to the outputting device and at the same time to the control signal generating unit 
500. 

If desired, to improve this decision making process, a number of detection 

1 5 windows may be considered. In such an instance, the false positive probability is a 

combination of the individual probabilities for each detection window considered, dependent 
upon the desired criteria. For instance, it could be determined that if the correlation function 
has two peaks above a threshold of cL = 7 on any two out of three detection intervals, then 
the watermark is deemed to be present. Obviously, such detection criteria can be altered 

20 depending upon the desired use of the watermark signal and to take into account factors such 
as the original quality of the host signal and how badly the signal is likely to be corrupted 
during normal transmission. 

In summary of the general operation of this particular re-embedding process, 
the re-embedding apparatus 600 is arranged to receive a signal y ' oW containing a watermark at 

25 input 602. The signal y ' 0 w in this instance has been generated by the watermark embedding 
apparatus shown in figures 2 and 3, and includes a watermark comprising two circularly 
shifted versions w oW and w re fOi a single sequence w s of values. A copy of the received signal 
y 0 i d is passed to the detector 640. 

As described above, the detector 640 is arranged to detect the presence of the 

30 watermark within the signal y ' 0 w, and to estimate the watermark embedding parameters (e.g. 
the amounts d by which the watermark sequences were circularly shifted, and the gain factor 
a used to control the trade off between the audibility and the robustness of the watermark). 

Figure 10 illustrates a correlation function of the watermark embedded within 
y ' 0 id, with the original sequence of values (w s ) used to form w 0 id. As can be seen, the payload 
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pL oW of the watermark in y' oM is, at least in part, defined by the two amounts by which the 
sequences comprising the watermark have been circularly shifted, d 0 id and d Te f. 

In this preferred embodiment, only a portion of the original watermark signal 
is removed (the sequence of values which have been circularly shifted by the amount d 0 u)- 
5 The same sequence of values (w s ) as utilised in the original watermark is then circularly 
shifted by a new amount (d new ), and embedded within the information signal, using the 
detected embedding parameters. 

Figure 1 1 illustrates a correlation function of the same watermark signal 
shown in figure 10, but in which the sequence of values corresponding to delay d 0 u has been 
1 0 removed (i.e. as if "-w oW " had been added to y ' oU ). The result is a single correlation peak at 

d re f. 

Figure 1 1 shows the correlation function of the same signal shown in Figure 
10, but after the same sequence of values w s with a new circular shift delay (d new ) has been 
inserted. This results in two correlation peaks, separated by a payload/?I new (see equation 2 

15 and 4) i.e. a new watermark signal. It will be appreciated that by only removing one half of 
the original watermark signal, and subsequently adding in a replacement half of the 
watermark signal, a new watermark has been generated but with minimum impact upon the 
quality of the information signal into which the watermark is embedded. 

The detector 640 also provides synchronization information such as the time 

20 offset (At) to a delay unit 629. The copy of the input signal y ' oW passed to the delay unit 629 
is then appropriately synchronized to take into account any time offsets, rescaling and also 
the intrinsic delay caused by the various operations carried out by the units in the re- 
embedder (e.g. 630b, 624, 625, 615), and to ensure that y ' old is synchronized with y b at the 
adder 626. One copy of the signal y ' 0 id is passed to the filter H 615. This is similar to the 

25 filter shown in figure 7, and can be omitted. Such a filter, can for instance, be a band-pass 
filter, and gives an output v^. 

It will be appreciated that the above embodiments are provided by way of 
example only. Various modifications will be apparent to the skilled person. 

For instance, whilst the preferred embodiment has described the partial 

30 removal of the original watermark, it will be appreciated that the whole of the original 
watermark could be removed and replaced. Equally, whilst only one sequence has been 
described as being removed and replaced by one sequence, it will be appreciated that a single 
sequence could be replaced by two or more sequences. Alternatively if the original payload 
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comprised three circularly shifted sequences or more, two such sequences could be replaced 
by a single sequence, or a plurality of sequences. 

Whilst the new watermark has been described as utilizing the same values of 
embedding parameters as originally used, the new watermark could of course use alternative 
5 values for any one of more of the embedding parameters. 

For instance, whilst the above embodiment has described the watermark 
altering signal w c as being scaled by a factor a, it will be appreciated that the two 
components forming w c (ie. w 0 id and w«e»v) could be scaled by different amounts before being 
added together, or before being separately added directly to the received signal y ' o!d . In order 
10 for the portion of the watermark to be removed, it is desirable that the embedding strength of 
the negative version of w oW is similar to that originally used to embed w 0 u in the host signal. 
However, w new can obviously be embedded into y ' 0 \d with any desired strength. 

It is desirable that watermarks are embedded into the host signal without 
unduly affecting the quality of the host signal. Preferably, all embedded watermark signals 
15 are imperceptible to an observer i.e. in an audio signal, the effect of the watermark signal can 
not be heard, or in a video signal, the effect of the watermark signal can not be seen. 

The ITU standard "Method for objective Measurements of Perceived audio 
quality", International Telecommunication Union, Geneva Switzerland (1999), defines a five 
grid scoring system (which is in conformation to ITU-R Rec BS.l 1 16 (rev. 1) (1997) and 
20 ITU-R Rec. BS.562-3 (1990) standards). The various scores are: 5 = Imperceptible, 4= 
Perceptible but not annoying, 3 = Slightly annoying, 2 = Annoying, 1 - Very annoying. 

Whilst it is preferable that all watermark signals are imperceptible, equally, a 
scoring of 4 on the ITU scale ("perceptible but not annoying") is acceptable in most systems. 

Whilst the above embodiment describes the implementation of the present 
25 invention with respect to one particular watermarking scheme, it will be appreciated that the 
invention can in fact be implemented using many other types of watermarking schemes. 

For instance, one type of audio watermarking scheme is to use temporal 
correlation techniques to embed the desired data (e.g. copyright information) into the audio 
signal. 

30 This technique is effectively an echo-hiding algorithm, in which the strength 

of the echo is determined by solving a quadratic equation. The quadratic equation is 
generated by auto-correlation values at two positions: one at delay equal to t, and one at 
delay equal to 0. In such a scheme, as echoes of the audio signal are added to the original 
audio signal, the resulting signal is in fact both an amplitude and a phase modulated version 
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of the original audio signal. At the detector, the watermark is extracted by determining the 
ratio of the auto correlation function at the two delay positions. 

Also known are watermarking schemes based on the amplitude modulation of 
DFT (Discrete Fourier Transform) co-efficients, that require the calculation of DFT's at both 
5 the encoder and the decoder. 

Similarly, WO 98/53565, US 6,175,627 and WO 00/00969 describe alternative 
techniques, to which the present invention could be applied, for embedding or encoding 
auxiliary signals (such as copyright information) into a multimedia host or cover signal. As 
detailed in WO 00/00969, a replica of the cover signal, or a portion of the cover signal in a 
10 particular domain (time, frequency or space), is generated according to a stego key, which 
specifies modification values to the parameters of the cover signal. The replica signal is then 
modified by an auxiliary signal corresponding to the information to be embedded, and 
inserted back into the cover signal so as to form the stego signal. 

At the decoder, in order to extract the original auxiliary data, a replica of the 
1 5 stego signal is generated in the same manner as the replica of the original cover signal, and 
requires the use of the same stego key. The resulting replica is then correlated with the 
received stego signal, so as to extract the auxiliary signal. 

Using an alternative embodiment of the present invention, the extracted 
auxiliary signal can be replaced by a new one. This can be achieved by appropriately 
20 subtracting the auxiliary information from the received signal using the stego key and the 
embedding parameters estimated using the detection unit. In relation to figure 1, this can be 
put into effect by utilizing a detector 640, a watermark generator 650, and a watermark re- 
embedder 620 responsive to the underlying embedding algorithm. 

It will be appreciated by the skilled person that various implementations not 
25 specifically described would be understood as falling within the scope of the present 
invention. For instance, whilst only the functionality of the embedding and detecting 
apparatus has been described, it will be appreciated that the apparatus could be realized as a 
digital circuit, an analog circuit, a computer program, or a combination thereof. 

Within the specification it will be appreciated that the word "comprising" does 
30 not exclude other elements or steps, that "a" or "an" does not exclude a plurality, and that a 
single processor or other unit may fulfil the functions of several means recited in the claims. 



WO 03/102947 PCT/IB03/02319 



1 . A method of processing a multimedia signal comprising a watermark signal, 
the method comprising the steps of: 

removing at least a portion of an original watermark signal; and 

adding a new watermark signal to the multimedia signal so as to form a new watermarked 
5 multimedia signal. 

2. A method as claimed in claim 1 wherein said original watermark signal is 
removed by applying a negative version of said original watermark signal to the multimedia 
signal. 

10 ; " 

3. .v» ;i )V,;'a method as claimed in claim 1, further comprising the step of determining 

the value of at least one of the parameters used to embed the original watermark in the 
multimedia signal. 

15 4, A method as claimed in claim 3, wherein said value of said parameter is 

utilized to remove at least a portion of said original watermark signal. 

5. A method as claimed in claim 3, wherein said new signal is embedded in the 
multimedia signal using said embedding parameters having said determined value. 

6. A method as claimed in claim 3, wherein said new signal is embedded in the 
multimedia signal using said embedding parameters having values other than the said 
determined values. . 



25 



7. A method as claimed in claim 3, wherein said parameter comprises at least one 

of: embedding strength, synchronization information, time offset, time-scaling, an amount of 
a circular shift of a sequence, and a watermark symbol period. 
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8. A method as claimed in claim 1, wherein all of said original watermark is 
removed, and the new signal comprises a new watermark signal. 

9. A method as claimed in claim 1, wherein said watermark comprises at least 

5 two sequences of values, at least one sequence of values being removed as said portion of the 
watermark signal, so as to leave at least one remaining sequence of values from the original 
watermark signal; and 

wherein said new signal comprises at least one further sequence of values which together 
with said remaining sequence forms anew watermark signal. 

10 

10. A method as claimed in claim 9, wherein all of said sequences of values are 
formed from a single sequence of values which has been circularly shifted by different 
amounts. 

15 11. A method as claimed in claim 1 , wherein said removed portion of the 

watermark signal was embedded with a predetermined strength into said multimedia signal, 
the method comprising the step of embedding the new signal into the multimedia signal with 
preferably the same predetermined strength. 

20 12. A method as claimed in claim 1 1 , wherein the embedding strength is such that 

the degradation in the quality of the new watermarked multimedia signal is perceptible but 
not annoying. 

13. A method as claimed in claim 1 , wherein at least one of the original watermark 
25 signal and the new watermark signal comprises a smoothly varying signal formed by 

applying a window shaping function to a sequence of values, the integral over the window 
shaping function being zero. 

14. A method as claimed in claim 13, wherein the window shaping function has a 
30 bi-phase behavior. 



15. A method as claimed in claim 14, wherein the bi-phase window comprises at 

least two Harming windows of opposite polarities. 
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16. A computer program arranged to perform the method of claim 1 . 



17. A record carrier comprising a computer program as claimed in claim 16. 

5 18. A method of making available for downloading a computer program as 

claimed in claim 16. 

19. An apparatus for processing a multimedia signal comprising a watermark 
signal, the apparatus comprising: 

1 0 a deletion unit arrange to remove at least a portion of the watermark signal; and 

an embedder arranged to add a new signal to the multimedia signal so as to form a new 
watermark signal. 

20. An apparatus as claimed in claim 19, further comprising a detector arranged to 
1 5 detect at least one value of a parameter of said watermark signal. 



21. 
19. 



A receiver of a multimedia signal comprising an apparatus as claimed in claim 
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